Improved Methods for Early Fault Detection in Enterprise Computing Servers Using SAS Tools

Advanced telemetry systems are being developed to collect and archive hundreds of system performance, throughput, quality-of-service (QoS), and physical variables for the purpose of enhancing the reliability, availability, serviceability, scalability, and security of business-critical enterprise computing servers. SAS software was chosen for this project because of the language's powerful coding features and its ability to model prototype systems quickly and costeffectively. SAS macro language was exploited in a parametric sensitivity study to explore and optimize clusters of signals that provide high sensitivity for early fault detection, but with a good avoidance of false alarms, for subsequent post-processing with an advanced statistical patternrecognition surveillance system called MSET. Finally, PROC GPLOT and PROC G3D proved indispensable for displaying results of our many-variable sensitivity investigation. SAS programs developed for this investigation are generic rather than operating system specific. The results presented in this paper used SAS 8.2 for Unix environments operating on Solaris 8 platforms. The coding practices discussed in this paper are aimed at users with an average SAS/GRAPH experience and an average proficiency with SAS macro language. Introduction Fault detection in complex systems typically requires costly on-line monitoring and expertise. Conventional approaches to identifying faults, combining event correlation and threshold-based rules, have proven inadequate in a variety of safety-critical industries with complex, heterogeneous subsystem inputs not dissimilar to those from enterprise computing. Fundamentally, while many high-end computing servers are already rich in instrumentation, the data produced by the instrumentation are complex, non-uniform, and difficult to correlate. Pattern recognition technology, coupled with continuous system telemetry, offers key enabling technology that can help: • identify incipient faults proactively • reduce the expertise required to identify correlations • reduce the compute cost of monitoring • reduce the probability of false alarms • increase the accuracy and timeliness of root cause analysis • help eliminate “No-Trouble-Found” (NTF) events that can drive up warranty & serviceability costs for a server vendor The effectiveness of using pattern recognition to discern incipient faults in noisy process data coupled with continuous system telemetry is gated by the quality of information available from instrumentation. This investigation was undertaken to extract, pre-process, and provide analytical resampling for a vast variety of performance, quality-of-service, and system load metrics for SUGI 29 Posters